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ABSTRACT 

Intelligent Tutoring Systems (ITSs) transcend 
conventional computer-based instruction. An ITS is 
capable of monitoring and understanding student 
performance thereby providing feedback, explanation, 
and remediation. This is accomplished by including 
models of the student, the instructor, and the expert 
technician or operator in the domain of interest. The 
space shuttle fuel cell is the technical domain for the 
project described below. 

One system, Microcomputer Intelligence for Technical 
Training (MITT), demonstrates that ITSs can be 
developed and delivered, with a reasonable amount 
of effort and in a short period of time, on a 
microcomputer. The MITT system capitalizes on the 
diagnostic training approach called Framework for 
Aiding the Understanding of Logical Troubleshooting 
(FAULT) (Johnson, 1987). The system’s embedded 
procedural expert was developed with NASA’s CLIPS 
expert system shell (Culbert, 1987). 


interface to the student user. The components of a 
generic ITS are shown in Figure 1 (Johnson, in press). 
There are many recent publications that offer thorough 
definitions of ITSs and describe existing systems 
(Psotka, Massey, & Mutter, 1988; Poison & 
Richardson, 1988; and Wenger, 1987). 
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Figure 1. Components of an Intelligent Tutoring 
System 


MITT was conceived and sponsored by the Air Force 
Human Resources Laboratory, Brooks Air Force Base, 
Texas. The research, development, and evaluation of 
MITT was completed with cooperation from NASA at 
the L.B. Johnson Space Center, Houston, Texas. 

INTRODUCTION 

Intelligent Tutoring Systems (ITSs) are instructional 
systems that deliver training in a manner comparable 
to that of a human tutor. ITSs deliver instruction, 
interact with students, and structure subsequent 
instruction based on student performance. To provide 
such instruction, ITSs must contain an understanding 
of a specific domain, a means to model student 
understanding of that domain, and a component 
containing pedagogical guidelines for providing 
feedback and remediation. These components must 
surround an instructional environment. The 
instructional environment must have a reasonable 


This paper describes a six-month research effort in 
which mature computer-based instruction (CBI) for 
diagnostic training was integrated with the NASA 
expert system shell, C Language Integrated 
Production System (CLIPS), to create a fully 
operational ITS on a microcomputer. The system is 
called Microcomputer Intelligence for Technical 
Training (MITT). 

Although ITSs have been around for nearly twenty 
years, they have not emerged as common 
components in training departments. Instead, they 
have remained as research topics in industry, 
government, and university laboratories (Johnson, in 
press). The purpose of the MITT project was to prove 
that ITSs can be developed in a reasonable amount of 
time for a reasonable price. Also, the MITT developers 
wanted to show that ITS technology is sufficiently 
mature to contribute to an operational training 
environment. 
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COMPUTER-BASED INSTRUCTION 

To develop an instructional environment for MITT, 
Search Technology, Inc. capitalized on its more than 
1 0 years of research experience with computer-based 
simulation for diagnostic training. That research (Hunt 
& Rouse, 1981; Johnson, 1981, 1987; Johnson & 
Rouse, 1981; Rouse and Hunt, 1984) was 
characterized by an evolving set of computer 
simulations and extensive experimental and real-world 
evaluations. This R&D was conducted in diverse 
domains, such as automotive mechanics, aviation 
mechanics, communication/electronics, and nuclear 
safety systems. Search Technology’s CBI research 
began as an attempt to understand how humans 
gather and process information in problem-solving 
situations. This led to research into the effects of 
training on problem-solving behavior. From this 
research, a variety of training concepts were 
developed and evaluated in laboratory and field tests. 
The diagnostic training simulations that emerged from 
this research were Troubleshooting by Application of 
Structural Knowledge (TASK) and Framework for 
Aiding the Understanding of Logical Troubleshooting 
(FAULT). Both simulations are described by Rouse and 
Hunt (1984). 

FAULT, proven in a variety of instructional domains, is 
at the heart of the MITT system. It uses a hard copy 


functional flow diagram along with an on-line display for 
student options, test results, and feedback. FAULT is 
a simulation that permits the user to engage in the 
same information processing that would take place 
during real equipment troubleshooting. This includes 
actions such as checking instruments, obtaining 
symptomatic reports from an operator, forming 
hypothesis, selecting tests, and identifying parts for 
replacement. FAULT simulations developed prior to 
MITT included a limited degree of intelligence that 
provided student advice and feedback (Johnson, 
Norton, Duncan, & Hunt, 1988). 

THE MITT SYSTEM 

The MITT system consists of five parts shown in Figure 
1 : the instructional environment, the student interface, 
the expert module, the student module, and the 
instructor module. This section describes each of the 
modules and how they communicate to form the 
Intelligent Tutoring System. 

The Instructional Environment 

MITT’s instructional environment is the FAULT 
simulation. Perhaps the greatest strength of the 
FAULT simulation is the simplicity of the system’s 
representation. Figure 2 shows MITT’s functional flow 
representation of the space shuttle fuel cell. Each part 
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Figure 2. MITT Fuel Cell Functional Flow Diagram 
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in the system is represented as a node in a network. 
MITT’s simple, single functional relationship helps the 
student create a mental model of the technical system. 

The Student Interface 

Although each module of an ITS is critically important, 
it is the user interface that ultimately delivers the 
instruction to the user. The interface must allow easy 
interaction with the simulation. Also, the outputs of the 
expert, student, and instructor modules must be 
completely integrated with the student interface in 
order to be effective and unobtrusive. 

A mouse is the primary user input device, although a 
keyboard may also be used at any time. The cursor, 
controlled by either the mouse or the keyboard, can be 
moved within each menu to select student diagnostic 
options. On certain screens, extensive data are 
available as shown in Figure 3. For the simulation to 
know precisely what data the student is requesting, it is 
necessary for the user to click on the XXX area (see 
column 3 in Figure 3) to obtain the information. 
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Figure 3. CRT Display Requiring Specific 
Information Request Areas 


The Expert Module 


procedures for the space shuttle’s fuel cell. At any 
stage during diagnosis of a malfunction, the student 
can request procedural advice. The PE gives advice 
according to the order in which gauges, controls, or 
panels have been seen by the student. Advice is 
given only when it is requested from the MITT menu by 
the student. This procedural advice suggests the next 
step to take in the diagnosis of a malfunction. The 
student may follow the advice that is given or choose 
from other menu options. 

PE development required an analysis of existing NASA 
troubleshooting procedures. These procedures were 
broken down into steps or actions that are likely to be 
used by the student. Each action corresponds to the 
reading of simulated gauges, controls, or panels 
available in the shuttle. Search Technology’s analysis 
of the MITT project had to ensure that the student had 
some means to accomplish a given NASA procedure 
within the constraints of the simulation. 

Each statement of advice is linked to a logical test. For 
example, if the orbiter’s primary annunciator panel (F7) 
has been seen and the front gauge panel (F9) or 
cathode-ray tube (CRT) display System Summary 1 
has been seen, then an advice statement describes 
what has been seen, what conclusions can can be 
drawn, and what actions are appropriate. All of the 
advice statements for the six malfunctions are 
incorporated into approximately 60 rules. Additional 
procedural intelligence can be incorporated by merely 
adding to the established rule base. 

The functional expert 

The functional expert (FE) is based on the functional 
connectivity among the system components. For 
example, if part B depends on part A and part A has 
failed, then part B will be adversely affected. Also, part 
B will adversely affect parts which depend on it. This 
functional understanding permits the expert to 
calculate how failures propagate through the system 
by following the functional topography. 


MITT’s expert module represents the system in two 
ways. It has both a procedural and a functional expert. 
The procedural expert (PE) maps symptoms to 
suspect systems and components with collections of if- 
then rules. These symptoms include information such 
as annunciator lights and instrument indications. The 
functional expert (FE) uses one or more connectivity 
matrices to represent various functional relationships 
among the parts of the system. 


The student has the option to interact with this expert 
by requesting FE advice while working on a problem. 
The FE provides assistance based on the functional 
structure of the system and the student’s previous 
actions. The FE also communicates with the student 
module to monitor changes in the feasible set of 
failures based on each student action. 

Expert integration 


The procedural expert 

The procedural expert is a series of rule-based 
statements. It contains information in the form of 
advice regarding specific NASA troubleshooting 


The experts must have the means to communicate 
with each other. During MITT’s development, this was 
accomplished by creating an equivalency table 
between gauge readings and topographic tests. For 
example, a reading of the 02/H2 flow on a gauge may 
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be the same as a test between two parts of the 
functional flow diagram. As far as MITT was 
concerned, they are the same. All of the tests were 
translated to gauge readings to match the format of the 
PE. In addition, the gauge readings were translated to 
functional flow tests for the FE. 

The Student Module 

The student module creates a model of the student by 
tallying the student’s actions throughout the simulation. 
The student model is updated by the FAULT simulation 
and the expert module. The data from the student 
model is used by the instructor module to determine 
when advice is appropriate. 

MITT has a student model that is current for each 
problem. The model includes tests the student has 
taken and results of these tests. The model keeps 
track of the number of times the student uses an 
option from the primary simulation display as well as 
the number of accesses to each orbiter gauge, 
annunciator, or CRT. The model also keeps track of 
the type of errors noted by the functional expert. The 
student module provides feedback upon completion of 
each problem as shown in Figure 4. 


Congratulations! You have corrected the problem. 
Number of students completing this problem: 

78 


Number of students who quit before solving: 

2 


Number of students who received a time out: 

6 



Your 

Average 

Number of minutes for diagnosis: 

7 

8.4 

Number of errors made: 

2 

1.6 

Number of displays accessed: 

14 

17.3 

Number of times procedural advice used 
Number of times functional advice used 

1 

1.8 

2 

1.3 


Figure 4. MITT Feedback at Problem Completion 


The Instructor Module 

The instructor module is a rule-based routine that 
pinpoints certain student errors and intervenes as they 
occur. The types of errors detected include student 
actions that result from a student misunderstanding of 
either the MITT system or of simple troubleshooting 
procedures. The advantage to this approach is that 
the instructor module provides generic advice that 
promises to be effective for current and future 
simulations. 

The instructor module was designed to help guide the 
student to the most appropriate segment of the MITT 
system. Instead of redundantly explaining something, 
it suggests where the student should look for more 
information. In this sense, the instrucor module 
actually works more like a reference librarian than a 


teacher. The advantage to this approach is that only 
those who need reference material use it. 

The entire MITT system is designed to deliver 
instruction in such a way that even novices can easily 
use the system. The system includes help for using 
the MITT simulation and contains technical information 
and diagnostic advice on the fuel cell’s domain. MITT 
is student driven with complete learner control. Only 
under exceptional circumstances does the instructor 
module intervene and redirect the student from 
exploring his own paths. 


Hardware 

The hardware system used for the development and 
delivery of the MITT ITS was an IBM-AT (or an IBM- 
compatible system). The system requires 640 Kb of 
random access memory (RAM) and a hard disk. 
Presently, an Color Graphics Adapter (CGA) card and 
color monitor are needed. A mouse is recommended, 
but optional. 

The rationale for using IBM-ATs is straightforward. 
First, the equipment is affordable and more likely to be 
found in research laboratories and (more importantly) 
training installations. When a group decides to use 
MITT, they will not have to purchase expensive, 
dedicated Al workstations. Second, IBM-ATs have all 
the necessary speed, storage, hardware and software 
support, and other capabilities to deliver the required 
level of intelligent tutoring. In addition, IBM-ATs offer 
readily available off-the-shelf peripherals for interface to 
video disk and other computers. 


CLIPS 

MITT is written in C. By capitalizing on CLIPS, MITT 
processes rules for the various modules without using 
LISP. Therefore during MITT’s development, CLIPS 
was a convenient tool to use for several reasons. It 
can be embedded into existing C code, it has an built- 
in inference engine, the rules are simple to create, and 
CLIPS itself is easy to learn. In addition, CLIPS is 
highly portable across computer systems. CLIPS runs 
on IBM-ATs and IBM compatibles, and it is inexpensive 
(i.e., free for government use). 

The disadvantage to using CLIPS, during MITT’s 
development, was that Search Technology did not 
have a reliable, compiled version of CLIPS at the time 
MITT was completed. The uncompiled CLIPS version 
caused a delay in the system’s response time to the 
student. The new, compiled version of CLIPS will 
increase the speed with which procedural advice is 
given, thereby improving user acceptance of MITT. 



MITT EVALUATION 

Evaluation can be divided into two stages: formative 
and summative evaluations. Formative evaluation 
takes place during software design and development. 
Summative evaluation refers to the software’s value 
following development. For this short effort, the 
formative evaluation was most important. 

Formative Evaluation 

The primary goal of formative evaluation is to keep 
potential users informed as to ongoing development 
and the expected final product. Formative evaluation 
permits developers, subject matter experts, and 
prospective users to be constantly informed and able 
to make real-time changes in design. It also prevents 
the notion that "it is too late to change that now." 

The majority of MITT’s formative evaluation was 
accomplished by the ongoing interaction between the 
developers and personnel from NASA and AFHRL. 
During the program’s development, Search 
Technology followed a proven evaluation plan for CBI 
development (Maddox & Johnson, 1986). The plan 
ensured that software evaluation was performed using 
a three-step process: measuring compatibility, 

understandability and effectiveness. Compatibility 
refers to the extent to which the user is able to see the 
computer displays and reach the pointing devices. 
Displays must be legible, and all colors must be easily 
discernible. Understandability is concerned with ITS 
output and required user input. Users must be able to 
understand what the system is telling them and what 
they must tell the system. In addition, the system input 
requirements must aligned with the student’s prior 
knowledge and training. Effectiveness is similar to 
summative evaluation which is explained below. 


Summative Evaluation 

Summative evaluation takes place once the software is 
complete. It is a summary concerned with the effect of 
the training on student performance. MITT is not ready 
for a classic summative evaluation because only the 
first 6-month phase of the work is finished. However, 
the first phase did undergo two pseudo-summative 
evaluations that served as a way to define the steps 
needed for the next phase of MITT’s development, 
tentatively called MITT II. 

The first stage of the summative evaluation was 
conducted by Search Technology and NASA. This 
evaluation took place over a 2-day period at the 
Johnson Space Center. During that time, 17 NASA 
employees used MITT. These people included 
astronauts, flight controllers, CBI developers, Al 


researchers, technical instructors, and a training 
manager. The goal of the evaluation was to obtain a 
preliminary assessment of user acceptance and to 
insure that the ITS was complete and technically 
correct. User acceptance was overwhelmingly 
positive. 

The second stage of the summative evaluation was 
conducted by AFHRL and NASA, again at Johnson 
Space Center. For this evaluation, MITT was used by 
15 flight controllers for approximately 3 hours each. 
Again, the MITT ITS simulation was well received by 
the students. Many users commented that they saw 
value in the problem-solving approach to training 
presented by MITT. The users were impressed with 
the fact that they could proceed at their own pace with 
their own problem-solving style. The students were 
also satisfied with the advice they received from the 
functional, procedural, and instruction experts. 

The second evaluation pointed out areas of the MITT 
system that could be improved in subsequent versions 
of MITT. For example, it would be worthwhile to 
modify the on-line shuttle schematics to more closely 
resemble the functional flow diagrams. Some users 
commented that they would prefer higher fidelity 
displays of shuttle instrumentation. In addition, the 
current MITT problem knowledge base must be 
embellished to include a greater variety and number of 
problems. 

Comments from NASA’s instructional staff and training 
managers have been very positive. NASA is anxious to 
proceed with MITT ll’s development. Their first 
concern is to enhance the fuel cell knowledge base so 
that it can provide daily operational training to 
supplement existing classroom and simulator training. 
They also want the capability to build knowledge 
bases for additional space shuttle and space station 
subsystems. A goal of the MITT II program is to 
develop software tools and knowledge engineering 
techniques that will place an increasing amount of ITS 
development in the hands of training department 
personnel. These tools will help to decrease the costs 
of ITS development and further increase ITS 
availability. 


CONCLUSION 

MITT has clearly demonstrated that Intelligent Tutoring 
Systems can be developed in a reasonably short 
period of time at a reasonable cost. MITT has also 
shown that ITSs can be developed and delivered using 
off-the-shelf microcomputers. Most importantly, the 
MITT project has demonstrated that ITSs do not have 
to remain ’’laboratory rats", but instead can be viable 
components of operational training departments. 
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